Device, control method, and storage medium

ABSTRACT

A control method for generating a high quality HDR image is provided. A control device of the technique of this disclosure is a control device configured to control an image capturing element capable of controlling an exposure condition for each of areas, the device including: an acquisition unit configured to acquire an exposure value map obtained by preliminary exposure using the image capturing element; and a setting unit configured to set the exposure condition including a shutter speed and an ISO sensitivity for each of the areas based on the exposure value map.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a Continuation of International Patent ApplicationNo. PCT/JP2018/046254, filed Dec. 17, 2018, which claims the benefit ofJapanese Patent Application No. 2017-241983, filed Dec. 18, 2017, bothof which are hereby incorporated by reference herein in their entirety.

BACKGROUND OF THE INVENTION Field of the Invention

The technique of this disclosure relates to a device, a control method,and a storage medium capable of acquiring a high dynamic range image.

Background Art

There is conventionally known a technique of generating a high dynamicrange image (hereinafter referred to as “HDR image”) with a wide dynamicrange in a digital camera, a digital video camera, and the like. PTL 1discloses a method of acquiring an HDR image by one time image capturingby image capturing an object using an image capturing element capable ofcontrolling different exposure conditions for each pixel. According tothe method disclosed in PTL 1, an image multivalued by applying alow-pass filter to an image binarized using luminance is used as anexposure time map, thereby preventing the occurrence of a false contourat the boundary between pixels of different exposure times.

In the case of setting different exposure times for each pixel in imagecapturing, a long exposure time is set for a pixel corresponding to adark object. Depending on exposure conditions, however, the longexposure time may cause a blur in the acquired HDR image.

The technique of this disclosure has been accomplished in view of theabove problem and aims to generate a high quality HDR image in the caseof setting different exposure conditions for each area in imagecapturing.

CITATION LIST Patent Literature

PTL 1 Japanese Patent Laid-Open No. 2011-004088

SUMMARY OF THE INVENTION

A device according to the technique of this disclosure is a deviceconfigured to control an image capturing element capable of controllingan exposure condition for each of areas, the device comprising: anacquisition unit configured to acquire an exposure value map obtained bypreliminary exposure using the image capturing element; and a settingunit configured to set the exposure condition including a shutter speedand an ISO sensitivity for each of the areas based on the exposure valuemap.

Further features of the technique of this disclosure will becomeapparent from the following description of an exemplary embodiment to begiven with reference to the attached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic diagram showing an example of the appearance of animage capturing device in a first embodiment;

FIG. 2 is a diagram showing an example of the hardware configuration ofthe image capturing device in the first embodiment;

FIG. 3 is a block diagram showing an example of the functionalconfiguration of an image processing unit in the first embodiment;

FIG. 4 is a flowchart showing an example of a main procedure in thefirst embodiment;

FIG. 5 is a diagram showing an example of a UI of the image capturingdevice in the first embodiment;

FIG. 6A is a schematic diagram showing an example of a scene to becaptured in the first embodiment;

FIG. 6B shows an example of an exposure value map generated in an LDRimage capturing mode;

FIG. 6C shows an example of an exposure value map generated in an HDRimage capturing mode;

FIG. 7 is a flowchart showing an example of a correction procedure of anexposure condition in the first embodiment;

FIG. 8A is a diagram showing a specific example of exposure conditionsin a correction target area;

FIG. 8B is a diagram showing a specific example of exposure conditionsin the correction target area;

FIG. 8C is a diagram showing a specific example of corrected exposureconditions in the correction target area;

FIG. 9 is a diagram showing an example of a UI for adjusting the amountof correction in the first embodiment;

FIG. 10 is a block diagram showing an example of the functionalconfiguration of an image capturing device in a second embodiment;

FIG. 11 is a flowchart showing an example of a correction procedure ofan exposure condition in the second embodiment;

FIG. 12A is a diagram showing an example of an exposure value map beforesetting a correction target area; and

FIG. 12B is a diagram showing an example of an exposure value map aftersetting a correction target area.

DESCRIPTION OF THE EMBODIMENTS First Embodiment

In the first embodiment, a description will be given of a method ofgenerating an HDR image by controlling exposure conditions for eacharea. In particular, both of an exposure time (shutter speed) and an ISOsensitivity (analog gain) are controlled for each area, therebyperforming image capturing depending on the brightness of each area.

Configuration of Image Capturing Device

FIG. 1 is a schematic diagram showing an example of the appearance of animage capturing device in the present embodiment. An image capturingdevice 100 comprises an optical unit 101, an image capturing button 102,a display unit 103, and an operation button 104. The optical unit 101includes a zoom lens, a focus lens, a blur correction lens, a diaphragm,and a shutter to concentrate light information on an object. The imagecapturing button 102 is a button for accepting an image capturinginstruction from a user. The display unit 103 includes a liquid crystaldisplay or the like to display image data processed by the imagecapturing device 100 and various types of data. The operation button 104functions as an operation unit for accepting various instructions from auser. For example, a user can input an exposure condition to the imagecapturing device 100 via the operation button 104.

Hardware Configuration of Image Capturing Device

FIG. 2 is a block diagram showing an example of the hardwareconfiguration of the image capturing device 100 in the presentembodiment. An image capturing element unit 201 is a group of imagecapturing elements that convert light concentrated by the optical unit101 into a current value. The image capturing device 100 can acquirecolor information by using the image capturing element unit 201 incombination with a color filter and the like. In the present embodiment,an HDR sensor capable of setting exposure conditions for each pixel oreach area is applied to the image capturing element unit 201. In thepresent embodiment, “exposure condition” is a general term of parametersconcerning the brightness of an image. The image capturing element unit201 acquires an HDR image by adaptively controlling exposure conditions(such as an exposure time and analog gain) for each pixel or each area.In the case of controlling exposure in units of areas, exposureconditions are generally controlled for each 2×2 or 3×2 pixel area, butthe embodiment is not limited to this.

A CPU 202 has control over each constitutional element of the imagecapturing device 100 and sequentially reads commands stored in a storagearea such as a read only memory (ROM) 203 into a random access memory(RAM) 204. The CPU 202 interprets the read command and executesprocessing according to the result of interpretation. An image capturingsystem control unit 205 performs control according to an instructionfrom the CPU 202 such that, for example, the optical unit 101 focuses,opens the shutter, or adjusts the aperture. A device control unit 206performs control according to a user instruction accepted via the imagecapturing button 102 such that, for example, the image capturing device100 starts or finishes image capturing operation. The device controlunit 206 also performs control according to a user instruction acceptedvia the operation button 104 such that, for example, the display unit103 displays a predetermined operation screen. A graphic generation unit207 functions as a display control unit of the image capturing device100 and generates an image signal indicating a character, figure, imageor the like to be displayed on the display unit 103. An A/D conversionunit 208 converts the amount of light of an object detected by the imagecapturing element unit 201 into a digital signal. An image processingunit 209 processes the digital signal converted by the A/D conversionunit 208, thereby processing image data corresponding to the digitalsignal. An encoder unit 210 converts the image data processed by theimage processing unit 209 into a file format such as jpeg. Aninput/output interface (“interface” will be hereinafter referred to as“I/F”) 211 is an I/F used to transmit/receive image data to/from anexternal device such as a PC or various storage media (such as a harddisk, memory card, CF card, and SD card). The constitutional elements ofthe image capturing device 100 described above are connected via asystem bus 212 so as to communicate with each other.

Functional Configuration of Image Capturing Device

FIG. 3 is a block diagram showing the functional configuration of theimage processing unit 209 in the present embodiment. The function ofeach block shown in FIG. 3 is implemented by the CPU 202 reading aprogram code stored in the ROM 203 into the RAM 204 and executing thecode. Alternatively, some or all of the functions of the blocks in FIG.3 may be implemented by hardware such as an ASIC or electronic circuit.The same applies to the block diagram subsequent to FIG. 3.

In the image capturing device 100 of the present embodiment, the imageprocessing unit 209 comprises an exposure value map generation unit 301,an exposure condition acquisition unit 302, a reference determinationunit 303, an area selection unit 304, and an exposure conditioncorrection unit 305. The exposure value map generation unit 301generates, according to image data sent from the A/D conversion unit 208in preliminary exposure, an exposure value map in which an exposurevalue is stored for each pixel. An exposure value is a parameter forcontrolling brightness recorded using an ISO sensitivity and an exposuretime. The exposure condition acquisition unit 302 reads exposureconditions corresponding to the exposure value and acquires exposureconditions for each area. The reference determination unit 303determines reference exposure conditions based on a plurality ofexposure conditions. The area selection unit 304 selects a correctiontarget area for which exposure conditions are to be corrected from amongareas partitioned in preliminary exposure. The exposure conditioncorrection unit 305 corrects the exposure conditions for the correctiontarget area based on the reference exposure conditions.

Main Procedure

FIG. 4 is a flowchart showing a main procedure of the image capturingdevice 100 in the present embodiment. The processing in the flowchartshown in FIG. 4 is performed by the CPU 202 reading a program codestored in the ROM 203 into the RAM 204 and executing the code. In thedescription below, sign “S” indicates a step in the flowchart. The sameapplies to the flowcharts subsequent to FIG. 4.

In S401, the device control unit 206 sets exposure conditions such as alens aperture value, a shutter speed, and an ISO sensitivity accordingto a user instruction accepted via the operation button 104. In thepresent embodiment, the shutter speed and the ISO sensitivity, which arealso camera parameters, correspond to the exposure time and the analoggain, respectively. Exposure conditions other than these cameraparameters may be set according to an image capturing mode. FIG. 5 is adiagram showing an example of a user interface (“user interface” will behereinafter referred to as “UI”) of the image capturing device 100 inthe present embodiment. FIG. 5 shows an UI for setting an HDR imagecapturing mode, the UI being displayed on the display unit 103. A usercan select a desired image capturing mode by inputting an instructionvia the operation button 104. The image capturing mode set in S401 isnot limited to the HDR image capturing mode and may be an imagecapturing mode other than the HDR image capturing mode such as an imagecapturing mode according to the type of object such as a person orscenery or an image capturing mode according to a weather such as sunnyor cloudy.

In S402, the device control unit 206 determines whether the imagecapturing button 102 has been pushed. In a case where the imagecapturing button 102 has been pushed (S402: YES), the processingproceeds to S403. In a case where the image capturing button 102 has notbeen pushed (S402: NO), S402 is repeated.

In preliminary exposure in S403, exposure conditions for each area inthe image capturing element unit 201 are set. More specifically, theexposure value map generation unit 301 generates an exposure value mapin which an exposure value is stored for each pixel according to imagedata sent from the A/D conversion unit 208 in the preliminary exposure.The image processing unit 209 (exposure condition acquisition unit 302)of the present embodiment can read exposure conditions (FIG. 8A to FIG.8C) corresponding to an exposure value and acquire exposure conditionsfor each area. A description will be given of settings of exposureconditions for each area of the image capturing element unit 201 withreference to FIG. 6A to FIG. 6C.

FIG. 6A is a schematic diagram showing an example of a scene to becaptured in the present embodiment. In the scene to be captured in FIG.6A, a person in a room is captured by the image capturing device 100.The room where the person is present has a window and light enters fromoutside through the window.

FIG. 6B shows an example of an exposure value map generated in the caseof capturing the scene in a low dynamic range (LDR) image capturing modein the present embodiment. The LDR image capturing mode is a mode foracquiring a captured image with a dynamic range narrower than that inthe HDR image capturing mode and is applied, for example, in a casewhere “tone oriented” is selected in the UI shown in FIG. 5. In the LDRimage capturing mode, settings are made such that exposure isappropriate for the person. At this time, since an exposure value shownin the exposure value map in FIG. 6B is set for the entire scene to becaptured shown in FIG. 6A, overexposure occurs in the window areabrighter than the inside of the room where the person is present.

FIG. 6C shows an example of the exposure value map generated in the caseof capturing the scene in the HDR image capturing mode in the presentembodiment. The HDR image capturing mode is a mode for acquiring acaptured image with a dynamic range wider than that in the LDR imagecapturing mode and is applied, for example, in a case where “D rangeoriented” is selected in the UI shown in FIG. 5. At this time, theexposure value shown in the exposure value map in FIG. 6B is set for theindoor area including the person and an exposure value lower than thatfor the indoor area is set for the window area, whereby overexposure inthe window area can be suppressed.

Areas for which exposure values are to be set are not limited to those.For example, an exposure value may be set for each pixel as shown inFIG. 6C. Alternatively, an exposure value may be set for a relativelylarge area such as the area corresponding to the window.

Returning to the flowchart of FIG. 4, in S404, the exposure conditionsset in the preliminary exposure in S403 are corrected so as to reducedifferences in exposure conditions between the areas.

Correction Procedure of Exposure Conditions

FIG. 7 is a flowchart showing an example of a correction procedure ofexposure conditions in S404 in the present embodiment. The exposurecondition correction processing will be described in detail withreference to the flowchart of FIG. 7.

In S701, the exposure condition acquisition unit 302 acquires exposureconditions for each area from the exposure conditions set in S401 andthe exposure value map generated in S403.

In S702, the reference determination unit 303 determines exposureconditions to be a reference (hereinafter referred to as “referenceexposure conditions”) based on the exposure conditions acquired in S701.In the present embodiment, the reference exposure conditions determinedin S702 are a shutter speed and an ISO sensitivity. As the referenceexposure conditions, for example, exposure conditions corresponding toan area of a main object in a captured image obtained by capturing thescene to be captured of FIG. 6A may be selected.

The method of determining the reference exposure conditions is notlimited to the method of selecting exposure conditions corresponding toan area of a main object, provided that the reference exposureconditions can be suitably selected according to a scene to be captured.For example, in the case of using a focused area as a reference,exposure conditions corresponding to the focused area can beautomatically determined as the reference exposure conditions. In thecase of accepting selection of a desired area via the operation button104, exposure conditions corresponding to the desired area can bedetermined as the reference exposure conditions. Alternatively, thereference determination unit 303 can determine exposure conditionscorresponding to an arbitrary area as the reference exposure conditionsbased on the exposure conditions acquired in S701.

In S703, the area selection unit 304 selects a correction target areafor which exposure conditions are to be corrected from among areaspartitioned in the preliminary exposure in S403. The area selection unit304 can select the correction target area in an arbitrary order eachtime the loop processing from S703 to S705 is executed. For example, thearea selection unit 304 may select the correction target area whilescanning an area of interest in the order from upper left to lower rightof the captured image. At this time, in a case where a differencebetween the reference exposure condition and exposure condition for thecorrection target area is small, the advantageous result obtained bycorrecting the exposure condition also becomes small. Thus, an area ofinterest may be selected as the correction target area in a case wherethe difference between the reference exposure condition and the exposurecondition for the area of interest is larger than a predeterminedthreshold.

In S704, the exposure condition correction unit 305 corrects theexposure conditions for the correction target area selected in S703based on the reference exposure conditions determined in S702. Theexposure condition correction processing will be described in detailwith reference to FIG. 8A to FIG. 8C.

FIG. 8A and FIG. 8B are tables showing specific examples of exposureconditions set in S401 for a correction target area. On the other hand,FIG. 8C is a table showing a specific example of exposure conditionscorrected in S704 for a correction target area. The exposure conditionacquisition unit 302 can acquire exposure conditions corresponding to anexposure value of each area (pixel) stored in the exposure value map byreferring to the tables of FIG. 8A and FIG. 8B. For example, adescription will be given of a case where the shutter speed “ 1/100 sec”and the ISO sensitivity “400” associated with the exposure value “0” areset as reference exposure conditions.

FIG. 8A shows an example of adjusting exposure conditions by changingonly the shutter speed. Since the exposure value of the person area is“0” in the scene to be captured of FIG. 6A, exposure conditions set forthe person area are the shutter speed “ 1/100 sec” and the ISOsensitivity “400.” In contrast, since the exposure value of the windowarea is “−2,” exposure conditions set for the window area are theshutter speed “ 1/400 sec” and the ISO sensitivity “400.” At this time,the shutter speed differs four times between the person area and thewindow area. Accordingly, in a case where the scene to be captured ofFIG. 6A includes a moving object, the amount of movement also differsfour times between the areas. As a result, in a case where the exposurevalues are adjusted based on only the shutter speed, a blur occurs insome local areas. In addition, the aspect of output such as a blur ormultiple overlapping image may largely differ between local areas in anHDR image.

FIG. 8B shows an example of adjusting exposure conditions by changingonly the ISO sensitivity. Since the exposure value of the person area is“0” in the scene to be captured of FIG. 6A, exposure conditions set forthe person area are the shutter speed “ 1/100 sec” and the ISOsensitivity “400.” In contrast, since the exposure value of the windowarea is “−2,” exposure conditions set for the window area are theshutter speed “ 1/100 sec” and the ISO sensitivity “100.” At this time,the ISO sensitivity differs four times between the person area and thewindow area. Accordingly, in a case where the scene to be captured ofFIG. 6A includes a moving object, the amount of noise also differs fourtimes between the areas. As a result, in a case where the exposurevalues are adjusted based on only the ISO sensitivity, the amount ofnoise increases in some local areas. In addition, the aspect of outputof noise may largely differ between local areas in an HDR image. Inother words, a difference in level of noise may be observed betweenlocal areas.

FIG. 8C shows an example of exposure conditions corrected by theexposure condition correction unit 305. Since the exposure value of theperson area is “0” in the scene to be captured of FIG. 6A, exposureconditions set for the person area are the shutter speed “ 1/100 sec”and the ISO sensitivity “400.” In contrast, since the exposure value ofthe window area is “−2,” exposure conditions set for the window area arethe shutter speed “ 1/200 sec” and the ISO sensitivity “200.” At thistime, a difference between the reference shutter speed “ 1/100” and thecorrected shutter speed “ 1/200” to be set for the person area isreduced to two times. Similarly, a difference between the reference ISOsensitivity “400” and the corrected ISO sensitivity “200” to be set forthe window area is also reduced to two times. In this manner, theexposure condition correction unit 305 controls not either but both ofthe shutter speed and the ISO sensitivity, thereby reducing the adverseeffect of controlling only either one of these conditions. Further,correcting the shutter speed and the ISO sensitivity at the same orsubstantially the same rate enables a reduction in a difference inaspect of appearance of a blur or noise between local areas. Inaddition, since correction is made not only to exposure conditionscorresponding to the boundary between the correction target area and theadjacent area but to exposure conditions corresponding to the entirecorrection target area, a false contour can also be reduced in an HDRimage. As a result, according to the control method of the imagecapturing device in the present embodiment, a high quality HDR image canbe generated in the case of setting different exposure conditions foreach area in image capturing.

The correction described above may be made according to a desired methodas long as differences between the reference exposure conditions andexposure conditions for each area can be reduced. For example, on theassumption that the exposure value in the reference exposure conditionsis E, the shutter speed is T, the ISO sensitivity is G, the exposurevalue of the correction target area is E′, the corrected shutter speedis T′, and the corrected ISO sensitivity is G′, T′ and G′ can becalculated by the following formulae, respectively:

T′=T*2^(α(E′-E))  (formulae 1)

G′=G*2^((1-α)(E′-E))  (formulae 2)

In the above formulae, a is a coefficient that is a real numbersatisfying 0.0≤α≤1.0. FIG. 8A corresponds to the case of α=1.0, FIG. 8Bcorresponds to the case of α=0.0, and FIG. 8C corresponds to the case ofα=0.5. The above formulae are not necessarily used and an arbitrarilyadjusted coefficient α may be used instead.

FIG. 9 is a diagram showing an example of a UI for adjusting the amountof correction in the present embodiment. As shown in the example of theUI in FIG. 9, a user can adjust the coefficient α by moving a knob 901of a slider bar included in the UI right or left. For example, in a casewhere the knob is moved right (direction of “sensitivity oriented”) toreduce a, a difference between the reference shutter speed and theshutter speed in the correction target area is reduced, whereby theoccurrence of a blur or multiple overlapping image caused by a movingobject can be more reduced. In contrast, in a case where the knob ismoved left (direction of “Tv oriented”) to increase a, a differencebetween the reference ISO sensitivity and the ISO sensitivity in thecorrection target area is reduced, whereby the occurrence of adifference in level of noise can be more reduced. In this manner, theexposure condition correction unit 305 can correct the shutter speed andthe ISO sensitivity at weighted rates, thereby reducing a difference inaspect of appearance of a blur or noise between local areas.

Returning to the flowchart of FIG. 7, in S705, the exposure conditioncorrection unit 305 determines whether the processing has been completedfor all the areas. In a case where the processing has been completed forall the areas (S705: YES), the processing returns to the flowchart ofFIG. 4. In a case where the processing has not been completed for allthe areas (S705: NO), the processing returns to S703 to perform the loopprocessing from S703 to S705.

Returning to the flowchart of FIG. 4, in S405, the image capturingsystem control unit 205 causes the image capturing system to performimage capturing operation based on the exposure conditions corrected inS404. In image capturing in S405, the image capturing system controlunit 205 drives the optical unit 101 to acquire the amounts of light ofobjects, and the acquired amounts of light are detected by the imagecapturing element unit 201. The A/D conversion unit 208 converts theamounts of light detected by the image capturing element unit 201 intoan electric signal to acquire RAW image data. As publicly known in thetechnical field of image processing, RAW image data is image data inwhich any one of R, G, and B color components is stored in each pixel ina predetermined arrangement such as the Bayer arrangement.

In S406, the image processing unit 209 performs development processingfor the RAW image data. By applying development processing to the RAWimage data, RGB image data (3-channel image data having all of the R, G,and B color components in each pixel) is generated. Although thedevelopment processing is generally accompanied with sub-processing suchas white balance processing, demosaicing processing, and gammaprocessing, the description thereof will be omitted since they are notthe main focus of the present embodiment.

In S407, the image processing unit 209 outputs the RGB image datagenerated in S406. The output RGB image data is sent to the encoder unit210 and converted into a file format such as jpeg. Next, the RGB imagedata is output to an external device or storage medium via theinput/output I/F 211. Upon the completion of S407, the flowchart isfinished.

As described above, according to the control method of the imagecapturing device of the present embodiment, in the case of settingdifferent exposure conditions for each area in image capturing, theshutter speed and the ISO sensitivity are controlled, thereby reducingthe adverse effect of controlling only either one of these conditions.Therefore, according to the control method of the image capturing deviceof the present embodiment, a difference in aspect of appearance of ablur or noise between local areas and a false contour can be reduced anda high quality HDR image can be generated.

Second Embodiment

In the correction processing of exposure conditions in the firstembodiment (S404), exposure conditions for the correction target areaare corrected based on the reference exposure conditions. However, in acase where a difference between the reference exposure condition and theexposure condition for the correction target area are small, theadvantageous result obtained by correcting the exposure condition alsobecomes small. Therefore, in the present embodiment, a difference inexposure condition between adjacent areas is considered. In case wherethere the difference is large, exposure conditions for an area ofinterest are corrected. The description of the same features as those ofthe first embodiment will be simplified or omitted and features uniqueto the present embodiment will be mainly described.

Functional Configuration of Image Capturing Device

FIG. 10 is a block diagram showing the functional configuration of theimage capturing device 100 in the present embodiment.

The image processing unit 209 of the present embodiment comprises anexposure value determination unit 1001 instead of the referencedetermination unit 303 in the first embodiment. The exposure valuedetermination unit 1001 determines whether an area of interest out ofareas partitioned in preliminary exposure is a correction target area tobe a target of correction.

Correction Procedure of Exposure Conditions

FIG. 11 is a flowchart showing the correction procedure of exposureconditions in S404 in the present embodiment. The exposure conditioncorrection processing will be described in detail with reference to theflowchart of FIG. 11.

First, in the loop processing from S1101 to S1104, a correction targetarea for which exposure conditions are to be corrected is determined.

In S1101, the exposure value determination unit 1001 selects an area ofinterest from among areas of the image capturing element unit 201partitioned in the preliminary exposure. The area of interest can beselected in an arbitrary order each time the loop processing from S1101to S1104 is executed. For example, the area may be selected in the orderfrom upper left to lower right in the captured image.

In S1102, the exposure value determination unit 1001 compares anexposure value of the area of interest selected in S1101 and an exposurevalue of an adjacent area adjacent to the area of interest. In a casewhere a difference in exposure value is equal to or larger than apredetermined threshold (S1102: YES), the processing proceeds to S1103.In a case where the difference in exposure value is smaller than thepredetermined threshold (S1102: NO), the processing skips S1103 andproceeds to S1104.

In S1103, the exposure value determination unit 1001 determines the areaof interest as a correction target area.

The method of determining the correction target area in the presentembodiment will be described with reference to FIG. 12A and FIG. 12B.

FIG. 12A shows an example of an exposure value map input to the exposurevalue determination unit 1001. FIG. 12B shows an example of an exposurevalue map output from the exposure value determination unit 1001. Thatis, in the exposure value map shown in FIG. 12B, shaded areas aredetermined as the correction target areas. In the example of FIG. 12B,areas adjacent to the area of interest in eight directions (upper left,above, upper right, left, right, lower left, below, and lower right) aresearched for. In a case where a difference in exposure value between thearea of interest and an adjacent area is equal to or larger than 2, thearea is determined as a correction target area. In this manner, based onthe exposure value map generated by the preliminary exposure, an arealargely different in exposure value from an adjacent area is determinedas a correction target area. The method of determining the correctiontarget area is not limited to above. Further, the difference in exposurevalue to be a determination condition may be other than “2.”

Next, in the loop processing from S1105 to S1107, exposure conditionsfor the correction target area are corrected.

Returning to the flowchart of FIG. 11, in S1105, a correction targetarea for which exposure conditions are to be corrected is selected fromamong areas partitioned in the preliminary exposure in S403. Thecorrection target area may be selected in an arbitrary order each timethe loop processing from S1105 to S1107 is executed. For example, thearea may be selected in the order from upper left to lower right in thecaptured image.

In S1106, the exposure condition correction unit 305 corrects exposureconditions for the correction target area selected in S1105. Since themethod of correcting the exposure conditions is the same as that in thefirst embodiment, the description thereof is omitted.

In S1107, the exposure condition correction unit 305 determines whetherthe processing has been completed for all the areas. In a case where theprocessing has been completed for all the areas (S1107: YES), theprocessing returns to the flowchart of FIG. 4. In a case where theprocessing has not been completed for all the areas (S1107: NO), theprocessing returns to S1105 and executes the loop processing from S1105to S1107.

As described above, according to the control method of the imagecapturing device of the present embodiment, in the case of settingdifferent exposure conditions for each area in image capturing, if adifference in exposure condition between adjacent areas is large,correction is made so as to reduce the difference in exposure conditionbetween the adjacent areas. Therefore, according to the control methodof the image capturing device of the present embodiment, a high qualityHDR image can be generated while reducing processing load caused bycorrection of exposure conditions, in addition to the advantageousresult of the first embodiment.

Other Embodiments

Embodiment(s) of the technique of this disclosure can also be realizedby a computer of a system or apparatus that reads out and executescomputer executable instructions (e.g., one or more programs) recordedon a storage medium (which may also be referred to more fully as a‘non-transitory computer-readable storage medium’) to perform thefunctions of one or more of the above-described embodiment(s) and/orthat includes one or more circuits (e.g., application specificintegrated circuit (ASIC)) for performing the functions of one or moreof the above-described embodiment(s), and by a method performed by thecomputer of the system or apparatus by, for example, reading out andexecuting the computer executable instructions from the storage mediumto perform the functions of one or more of the above-describedembodiment(s) and/or controlling the one or more circuits to perform thefunctions of one or more of the above-described embodiment(s). Thecomputer may comprise one or more processors (e.g., central processingunit (CPU), micro processing unit (MPU)) and may include a network ofseparate computers or separate processors to read out and execute thecomputer executable instructions. The computer executable instructionsmay be provided to the computer, for example, from a network or thestorage medium. The storage medium may include, for example, one or moreof a hard disk, a random-access memory (RAM), a read only memory (ROM),a storage of distributed computing systems, an optical disk (such as acompact disc (CD), digital versatile disc (DVD), or Blu-ray Disc (BD)™),a flash memory device, a memory card, and the like.

While the technique of this disclosure has been described with referenceto exemplary embodiments, it is to be understood that the invention isnot limited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures andfunctions.

According to the device of the technique of this disclosure, a highquality HDR image can be generated in the case of setting differentexposure conditions for each area in image capturing.

1. A device configured to control an image capturing element capable ofcontrolling an exposure condition for each of areas, the devicecomprising: an acquisition unit configured to acquire an exposure valuemap obtained by preliminary exposure using the image capturing element;and a setting unit configured to set the exposure condition including ashutter speed and an ISO sensitivity for each of the areas based on theexposure value map.
 2. The device according to claim 1, wherein thesetting unit sets the exposure condition corresponding to a relativelybright area at a value lower than a value of the exposure conditioncorresponding to a relatively dark area.
 3. The device according toclaim 1, further comprising a correction unit configured to correct theshutter speed and the ISO sensitivity of the exposure condition set foreach of the areas such that a difference in exposure condition is smallbetween at least some of areas in an image obtained by image capturingusing the image capturing element.
 4. The device according to claim 3,wherein the correction unit corrects each of the shutter speed and theISO sensitivity of the exposure condition set for each of the areas suchthat an amount of correction of an exposure value is equal orsubstantially equal.
 5. The device according to claim 3, furthercomprising a determination unit configured to determine a referenceexposure condition from among the exposure conditions set for therespective areas, wherein the correction unit corrects the shutter speedand the ISO sensitivity of the exposure condition set for each of theareas such that a difference from the reference exposure condition issmall.
 6. The device according to claim 5, wherein the determinationunit determines an exposure condition for a focused area as thereference exposure condition.
 7. The device according to claim 5,wherein the determination unit includes a display control unitconfigured to display, on a display unit, a UI for selecting an area ofan exposure condition to be the reference exposure condition.
 8. Thedevice according to claim 5, wherein in a case where a differencebetween the reference exposure condition and an exposure condition setfor an area of interest is equal to or larger than a predeterminedthreshold, the correction unit corrects a shutter speed and an ISOsensitivity of the exposure condition set for the area of interest suchthat the difference between the reference exposure condition and theexposure condition set for the area of interest is small.
 9. The deviceaccording to claim 5, wherein the reference exposure condition is atleast one of: a shutter speed and an ISO sensitivity set for an area ofa main object included in the image; a shutter speed and an ISOsensitivity set for an area focused in capturing the image; and ashutter speed and an ISO sensitivity set for an area selected via anoperation unit.
 10. The device according to claim 3, wherein in a casewhere a difference between exposure conditions set for adjacent areasout of the areas is equal to or larger than a predetermined threshold,the correction unit corrects the shutter speed and the ISO sensitivityof the exposure condition set for each of the areas such that thedifference between the exposure conditions of the adjacent areas issmall.
 11. The device according to claim 3, wherein the correction unitcorrects each of the shutter speed and the ISO sensitivity of theexposure condition set for each of the areas such that an amount ofcorrection of an exposure value is a predetermined weighted rate. 12.The device according to claim 3, further comprising a display controlunit configured to display, on a display unit, a UI for adjusting anamount of correction of the shutter speed and the ISO sensitivity of theexposure condition set for each of the areas.
 13. The device accordingto claim 1, further comprising an image capturing unit configured toperform image capturing according to the shutter speed and the ISOsensitivity of the exposure condition set for each of the areas.
 14. Thedevice according to claim 13, further comprising a development unitconfigured to perform development processing for image data acquired byimage capturing using the image capturing unit, wherein the developmentprocessing comprises white balance processing, demosaicing processing,and gamma processing.
 15. The device according to claim 1, furthercomprising a preliminary exposure unit configured to perform thepreliminary exposure for determining the exposure condition.
 16. Thedevice according to claim 1, wherein the setting unit comprises a tablestoring each of an exposure value, a shutter speed, and an ISOsensitivity and acquires an exposure condition corresponding to anexposure value of each area stored in the exposure value map.
 17. Acontrol method for controlling an image capturing element capable ofcontrolling an exposure condition for each of areas, the control methodcomprising: an acquiring step of acquiring an exposure value mapobtained by preliminary exposure using the image capturing element; anda setting step of setting a shutter speed and an ISO sensitivity foreach of the areas based on the exposure value map.
 18. A non-transitorycomputer readable storage medium storing a program for causing acomputer to execute each step in a control method for controlling animage capturing element capable of controlling an exposure condition foreach of areas, the method comprising: an acquiring step of acquiring anexposure value map obtained by preliminary exposure using the imagecapturing element; and a setting step of setting a shutter speed and anISO sensitivity for each of the areas based on the exposure value map.